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1 TITLE OF THE INVENTION 

2 Communication System Using Digital Watermark for Synchronising 

3 Multiplexed Text Data to Video Frames 

4 BACKGROUND OF THF TNVFNTTON 

5 Field of the Invention 

6 The present invention relates generally to video communication 

7 systems and more specifically to a copy-protect video communication 

8 system. The present invention is particularly useful for standard digital 

9 television broadcasting. 

10 Description of the Related Art 

11 For standard digital television broadcasting, it is contemplated to 



12 multiplex text data with a video signal using vertical blanking intervals. 

13 For this purpose, a number of text data are multiplexed in a video 

14 program and each text data stream is associated with a particular scries of 
is video frames, or "events" so that each text appears on the screen of 

16 television receivers simultaneously with the associated event. In order 

17 for television receivers to synchronise a text data stream with the 

18 associated event, it is necessary for the broadcasting station to transmit 

19 timing information along with a video signal, as disclosed in Japanese 
2n Laid -Open Patent Application 1 1 -2764 1 . However, the use of timing 

21 information represents a waste of resource of the video frequency 

22 spectrum. 

23 On the other hand, digital (or electronic) watermark is currently 

24 receiving attention because of its ability to detect illegal duplication of 

25 video programs and a number of techniques have been developed for 
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1 hiding (embedding) identifying information into programs to be copy- 

2 protected. 

3 SUMMARY OF THE INVENTION 

4 It is therefore an object of the present invention to provide a system 

5 and method for synchronising text data streams to "events" of digital 
ft video stream without using timing information. 

7 The stated object is obtained by embedding a digital watermark in 

s the transmitted video stream and detecting the digital watermark in the 

9 received video stream and using it as a timing signal for synchronising a 

10 text data stream to associated video frames, 

11 According to one aspect of the present invention, there is provided 

12 a digital video communication system comprising, at a source site, an 
is embedding circuit for embedding a digital watermark in a digital video 

14 stream to produce a watermarked digital video stream, and a multiplexer 

15 for multiplexing a text data stream with the watermarked digital video 

16 stream to produce a multiplexed signal. At a sink site, the system 

17 includes a demultiplexer for demultiplexing the mulLiplexed signal for 

18 recovering a watermarked digital video stream and a text data stream, A 

19 digital watermark detector is provided for detecting the digital 

20 watermark embedded in the recovered digital video stream. A 

21 synchroniser responds to the detected digital watermark for 

22 synchronising the recovered text data stream to the recovered video 

23 stream. 

24 According to a second aspect the present invention provides a 

25 digital video communication method comprising the steps of embedding 
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1 a digital watermark in a digital video stream to produce a watermarked 

2 digital video stream, multiplexing a text data stream with the 

3 watermarked digital video stream to produce a multiplexed signal, 

4 demultiplexing the multiplexed signal for recovering a watermarked 

5 digital video stream and a text data stream, detecting the digital 

6 watermark embedded in the recovered digital video stream and using the 

7 detected digital watermark as a timing signal for reading the text data 

8 from said storage medium, and synchronising the recovered text data 

9 stream with the recovered digital video stream in response to the detected 
! « *Q digital watermark. 

m 11 BRIEF DESCRIPTION OF THE DRAWINGS 

i II 12 The present invention will be described in further detail with 

Hj 13 reference to the accompanying drawings, in which: 

' ^ 14 Kg- 1 is a block diagram of a digital video communication system 

j 3 15 according to the present invention; 

Ul 16 Fig. 2 is a timing diagram of a video stream along with text data 

O * 7 which occur at such timing that write and read operations can be 

faB " 18 performed in an alternate fashion; 

19 Fig. 3 is a timing diagram of a video stream along with lexl data 

20 which occur at such timing that write and read operations are performed 

21 simultaneously by using a dual-mode memory; 

22 Fig. 4 is a block diagram of a first modified embodiment nf the 

23 present invention; 

24 Fig. 5 i> a block diagram of a second modified embodiment of the 

25 present invention; 
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1 Fig. 6 is a timing diagram of a video stream along with text data 

2 which occur at such timing that write and read operations are performed 

3 simultaneously for different text data by using switched memories; and 

4 Fig. 7 is a block diagram of a third modified embodiment of the 

5 present invention associated with Fig. 6, 

6 DETAILED DESCKiFllQ.N 

7 In Fig. 1/ a video communication system according to one 

8 embodiment of the present invention is comprised by a transmitter 10 

9 which transmits a composite digital video signal through a noisy 

10 environment such as a cable or a wireless transmission system (including 

11 satellite and terrestrial broadcasting) or a storage system such as video 

12 cassette recorder 20 to a receiver 30. 

13 At the transmitter 10, a digital audio stream is processed by an 

14 audio compression circuit 11 and a textual data stream is processed by a 

15 data compression circuit 1 2. A digital video stream is combined with a 

16 watermark signal in an embedding circuit 13 so that the watermark is 

17 embedded in a series of video images. A suitable method for the digital 

18 watermark is described in Japanese Laid-Open Patent Application 10- 

19 155151. The output of embedding circuit 13 is processed by a video 

20 compression circuit 14 using a known compression technique such as 

21 MPEG-2 to produce an MPEG-2 transport video stream, 

22 The outputs of the audio and data compression circuits 11 and 12 

23 are multiplexed with the MPEG-2 transport video stream by a 

24 multiplexer 15 to produce a composite video signal by inserting the 

25 textual data into the vertical blanking intervals of the video format and 
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1 the audio signal into the low frequency region of the spectrum of the 

2 composite video signal. 

3 The output of multiplexer 15 is encrypted by an encryption circuit, 

4 or encryptor 16. A forward error correction (FEC) encoder 17 is provided 
3 for converting the output of the encryptor 16 into a format that allows 

6 errors in the transmitted signal to be detected and corrected at the 

7 intended receive site. The output of the FEC encoder 17 is converted by 

& an interface IS to a radio frequency signal and transmitted to the receiver 

9 30 if the intermediate system 20 is a transmission system. If the system 20 

10 is a storage system, the output of the FEC encoder 17 ib directly supplied 

11 to the storage system and stored in a suitable storage medium. 

12 The signal from the transmitter 10 is received by an interface 31 at 

13 the receiver 30 to produce a replica of the output signal of the FF.C 

14 encoder 17. If the intermediate system 20 is a wireless link, the received 

15 signal is down-converted by the interface 31 to the baseband signal and if 

16 it is a storage system the stored signal is a baseband signal. 

17 The received baseband signal is applied to an FEC decoder 32 

18 where errors, if present in the received signal, are detected and corrected. 

19 The error-corrected signal is decrypted by a decryplor 33 and fed to a 

20 demultiplexer 34 to decompose the decrypted signal into the compressed 

21 audio, data and MPEG-2 transport video components, which are 

12 expanded to the original format by respective expansion circuits 35, 36 

23 and 37. 

24 The text data recovered by data expansion circuit 36 is supplied to 

25 a memory or synchroniser 37 and the MPEG-2 transport video stream 
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recovered by the video expansion circuit 38 is supplied to a watermark 
detector 39 to detect the watermark embedded in the video stream in a 
manner as disclosed in the aforesaid Japanese laid-open palenl 
application. 

The detected watermark signal is used by the memory 37 as a read 
timing signal for reading the stored text data so that it is synchronised to 
a aeries of desired video frames. The detected watermark is also supplied 
to a copy protect circuit 40 to produce a copy management signal that is 
used to protect video programs from being illegally copied by 
unauthorised users. Hie copy management signal is embedded in the 
video stream from video expansion circuit 38 as a digital watermark by 
an embedding circuit 41 lo protect a series of video frames from 
unauthorised duplications. 

The text data from the memory 37 and the output of embedding 
circuit 41 are supplied to a video display system 42. 

An example video stream is shown in Fig. 2 in which multiplexed 
text data 1 and 2 occur at such timing that the memory 37 can operate 
alternately in a write mode and a read mode. Two watermark signals are 
shown embedded in a video stream for synchronising text data 1 to a 
series of video frames 1 to 4 (event 1), and synchronising text data 2 to a 
scries of video frames 5 to 8 (event 2). When the video stream is 
displayed on the display system 42, the event-1 video frames 1 to 4 will be 
overlaid with text data 1 and the event-2 video frames 5 to 8 will be 
overlaid with text data 2. 

Preferably, the memory 37 is of a dual-mode type in which it 
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1 operates simultaneously in write and read modes As shown in Fig. 3, 

2 this mode of operation is useful when a read operation of text data 1 and 

3 2 is started while they are still written into the memory, 

4 In a mortified embodiment shown in Fig. 4, a video overlay unit 43 

5 is provided between the video expansion circuit 38 and the watermark 

6 embedding circuit 41 . Video overlay unit 43 receives the output of 

7 memory 37 to superimpose the text data on video frames specified by the 

8 detected digital watermark. 

9 In a further modified embodiment shown in Fig. 5, the copy 

10 protect circuit 40 is replaced with a watermark converter 44 in which the 

1 1 watermark signal detected by the watermark deter tor 39 is altered into a 

12 form that can be detected as a copy management signal to protect 

13 specified video frames from being illegally copied by unauthorised users. 

14 The output of watermark converter 44 is combined with the output of 

15 video overlay unit 43 in the embedding circuit 41. 

16 if text data 1 and 2 occur in sequence but the read timing of text 

17 data 1 coincides with the write timing of text data 2 as shown in Fig. 6, a 

18 modification of Fig, 7 is useful. In this modification, two graphic 

19 memories 61 and 62 are provided and a write control switch 63 is 
2U connected between the output of data expansion circuit 36 and the 

21 memories 61, 62 and a read control switch 64 is connected between the 

22 memories 61, 62 and the video overlay circuit 43. Memory control 

23 switches 63 and 64 are controlled by the output of the watermark detector 

24 39 so that when one of the memories 61, 62 is in a write mode the other 

25 memory is in a read mode. 
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Whatis claimed is: 

1 1 . A digital video communication system comprising: 

2 at a source site, 

3 an embedding circuit for embedding a digital watermark in a 

4 digital video stream to produce a watermarked digital video stream; and 

5 a multiplexer for multiplexing a text data stream with the 

6 watermarked digital video stream to produce <i multiplexed signal, 

7 at a sink site, 

8 a demultiplexer for demultiplexing the multiplexed signal for 

9 recovering said watermarked digital video stream and said text data 

10 stream; 

11 a digital watermark detector for detecting the digital watermark 

12 embedded in the recovered digital video slream; and 

1 3 a synchroniser for synchronising the recovered text data stream to 

14 the recovered digital video stream in response to the detection of said 

15 digital watermark. 

1 2. The digital video communication system of claim 1, further 

2 comprising a digital overlay circuit for superimposing the synchronised 

3 text data stream with the recovered digital video stream. 

1 3. The digital video communication system of claim 1, 

2 wherein said synchroniser comprises a memory, 

1 4. The digital video communication system of claim 1, 

2 wherein said synchroniser comprises a dual-mode memory. 
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1 5. The digital video communication system of claim 1/ 

2 wherein said synchroniser comprises a pair oi ftrst and second memories 

3 and a control circuit for alternately operating the first and aecond 

4 memories in write and read modes in response to the detected digital 
3 watermark. 

1 6 The digital video communication system of claim 5, further 

2 comprising a digital overlay circuit for superimposing the text data 

3 stream read out of each of said first and second memories with the 

4 recovered digital video stream, 

1 7. The digital video communication system of claim 1, further 

2 comprising at said sink site: 

3 a copy protect circuit responsive lo the detected digital watermark 

4 for producing a copy management signal; and 

5 an embedding circuit for embedding the copy management signal 

6 in the recovered digital vidpo .stream as a second digital watermark for 

7 preventing illegal duplication of the digital video stream. 

1 8. The digital video communication system of claim \, further 

2 comprising at said sink site: 

3 means for converting the detected digital watermark to a second 

4 digital watermark; and 

5 an embedding circuit for embedding the second digital watermark 

6 in the recovered digital video stream. 
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9. The digital video coixununicalion system of claim 1, further 
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2 comprising at said source site: 

3 a video compression circuit for compressing the watermarked 

4 digital video stream to supply an MPEG-2 transport stream to said 

5 multiplexer; 

6 an encryption circuit fur encrypting the multiplexed signal; and 

7 a forward error correction (FEC) encoder for encoding the 

8 encrypted signal; 

9 at said sink site, 

10 an FEC decoder for decoding the encoded signal to recover an 

n encrypted signal; 

12 a decryption circuit for decrypting the encrypted signal to recover 



13 a multiplexed signal of said MPEG-2 transport stream and said text data 

14 stream and supplying the multiplexed signal to said demultiplexer, 

15 whereby the MPEG-2 transport stream and said text data stream are 

16 individually recovered by said demultiplexer; and 

17 a video expansion circuit for expanding the MPEG-2 transport 

is stream recovered by said demultiplexer to supply a signal corresponding 

19 to said watermarked digital video stream to said watermark detector, 

1 10. A digital video communication method comprising the 

2 steps of: 

3 a) embedding a digital watermark in a digital video stream to 

4 produce a watermarked digital video stream; 

5 b) multiplexing a text data stream with the watermarked 

6 digital video stream to produce a multiplexed signal; 

7 c) demultiplexing the multiplexed signal for recovering a 
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8 watermarked digital video stream and a text data stream; 

9 d) detecting the digital watermark embedded in the recovered 

10 digital video stream and using the detected digital watermark as a timing 

11 signal for reading the text data from said storage medium; and 

12 e) synchronising the recovered text data stream with the 

13 recovered digital video stream in response to the detected digital 

14 watermark. 

1 11. The method of claim 10, wherein the step (e) comprises 

2 storing the recovered text data stream in a memory- and reading the 

3 stored text data stream from the memory in response to the darted 

4 digital watermark. 

1 12. A digital television receiver for receiving a digital video 

2 signal in which a digital video stream and a text data stream are 

3 multiplexed and Ihe digital video stream is watermarked by a digital 

4 watermark signal associated with the text data stream, comprising: 

5 a demultiplexer for demultiplexing die digital video signal for 

6 recovering the watermarked digital video stream and the text data 

7 stream; 

8 a digital watermark detector for detecting the digital watermark in 

9 the recovered digital video stream; and 

10 a synchroniser for synchronising the recovered text data stream to 

11 the recovered digital video stream in response to the detection of said 

12 digital watermark. 
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1 13. The digital television receiver of claim 12, further 

2 comprising a digital overlay circuit for superimposing the synchronised 

3 text data stream with the recovered digital video stream, 

1 14. The digital television receiver of claim 12, wherein said 

2 synchroniser comprises a memory. 

1 15. The digital television receiver of claim 12, wherein said 

2 synchroniser comprises a dual-mode memory. 

1 16. The digital television receiver of claim 12, wherein said 

2 synchroniser comprises a pair of first and second memories and a control 

3 circuit for alternately operating the first and second memories in write 

4 and read modes in response to the detected digital watermark. 

1 17. The digital television receiver of claim 16, further 

2 comprising a digital overlay circuit for superimposing the text data 

3 stream read out of each of said first and second memories with the 

4 recovered digital video stream. 

1 18. The digital television receiver of claim 12, further 

2 comprising; 

3 a copy protect circuit responsive to the detected digital watermark 

4 for producing a copy management signal; and 

5 an embedding circuit for embedding the copy management signal 

6 in the recovered digital video stream as a second digital watermark for 

7 preventing illegal duplication of the digital video stream. 
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1 19. The digital video communication system of claim 12, 

2 further comprising: 

3 means fox converting the detected digital watermark to a second 

4 digital watermark; and 

5 an embedding circuit for embedding the second digital watermark 

6 in thp recovered digital video stream. 
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ABfiTR ACT OF THE PISCLOSURE 

1 A digital video communication system includes, at a source site, an 

2 embedding circuil for embedding a digital watermark in a digital video 

3 stream to produce a watermarked digital video stream, and a multiplexer 

4 for multiplexing a text data stream with the watermarked digital video 

5 stream to produce a multiplexed signal. At a sink site, the system 

6 includes a demultiplexer tor demultiplexing the multiplexed signal for 

7 recovering a watermarked digital video stream and a text data stream, A 

8 digital watermark detector is provided for detecting the digital 

9 watermark embedded in the recovered digital video stream. A 

10 synchroniser responds to the delected digital watermark for 

11 synchronising the recovered text data stream to the recovered video 

12 stream. 
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